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Abstract ' . 

The problem of developing a comprehensive system of evaluation to improve 
an:i reward instructional effectiveness that would be acceptable to both the 
academic and administrative arms of an institution of higher educat^s^n has 
long bedeviled both. This paper reviews current practices and then outlines 
one possible approach with suggestions as to: (a) methods of assessing 
instructional effectiveness; (b) methods of providing results to the instructor; 
(c) options the instructor snould consider based on the evaluations; (d) hovy 
the instructional improvanent proposal should be initiated and then concluded 
i); a final report; and (e) an administrative procedure for incorporating 
instructional effectiveness assessments in the institutional reward system. 
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DEVELOPING A COMPREHENSIVE SYSTEM TO IMPROVE AfiD REWARD 

Cv 

INSTRUCTIONAL EFFECTIVENESS 

Lawrence M. Aleainoni 

More than anything else, an institution of higher education needs visible 
ways of convincing its faculty that: (a)^ good instructors are desired, (b) good 
instruction is respected, (c) thQ improvement of instruction is a major concern, 
(d) there are specific ways of recognizing and rewarding good instruction. Every 
iociti tution is already committing a major portion of its budget to maintain its 
instructional program: faculty salaries, classrooms, libraries, laboratories, 
instructional services, sabbatical leaves, and the like. Interestingly enough, 
however, when .a new graduate prepares to embark^on a professional career at any 
one of the institutions of higher education in the United States or abroad, he 
or she is told by the department head or' dean that the institution embraces the 
three general 'Objectives of excellence in teaching, research, and service and 
that rev;ards are based upon satisfactory to excellent performance in any one or 
a combination of those objectives. The sad fact is that after a short period of 
time at the institution, the new faculty member realizes that although the 
three objectives of teaching, research, and service are appropriate for any 
institution of higher education, most of the institutions reward faculty primarily 
for their performance in the research function (Astin & Lee, 1966; Thorne, Scott, 
X Beaird, 1976): 

This is -a disturbing state of affairs because it indicates, basically, 
that most institutions are only interested in supporting and encouraging 
excellence in research. Its consequences are obvious in that faculty, regardless 



hi thoir inttMH-.t:, fnay ruMilocL iheir tcMchintj and service octivitios in ordrM^ to 
attain the professional recognition reciuired to remain and succood at their 
inr.ti tution. -S-Ludofi ts are perhaps ttie most unfortunate pav;ns in vuich a cjaiiu? 
since they are forced to tai.e coiirses from faculty who are not able or willing 
to take the time to prepare and organize tliei r courses or to do such things as 
spenri f-.ime outside of class discussing problems and concerns that would help 
most students learn material hotter. 

F,ome take the position that there, is^" no inconsistency in this type of 
skewed reward system since it is suggested that excellent researchers are, 
in fact, the best teachers (Borgatta, 1970; Deming, 1972). The research evidence 
(Guthrie, 1949, 1954; Voeks, 1962; Hayes, 1971; Aleamoni, 1972b; Alcamoni & 
Yimer, 1973; Linsky & Straus, 1975), however, does not support this point and 
shows that, in general, there is no correlation between scholarly productivity 
and effective teaching. 

It is further suggested that the evaluation of scholarly productivity 
is- much easier, more valid, and more reliable than is the evaluation of instruc- 
tional effectiveness. This attitude has generally' resulted in a policy (dictated 
by practice) indicating that, regardless of the quality of the evaluative 
teaching evidence, published research will* still take precedence in the reward 
considerations. Under this policy the relationship between teaching and - 
scholarly productivity is believed to be high and positive even though, as was 
stated above, the rasearch evidence to date does not support this belief. 

Now let us look more closely at the criteria used to judge scholarly 
•achievement. They rest basically on the belief that judgments by colleagues 
(synonomous with "the academiE communi ty at large") provide the final evidence. 



TIkv.;o judfjinent:^, typically include: (a) obLaining ro'^enrch grants, (b) [)ublication 
in journials whore expert evaluation is required for acceptance, (c) favorable 
reviews of books, (cl) appointments or awards that require evaluation of pi^ofes- 
sional competence, (e) ^election to office in learned societies, and (f) rbceipt 
or fellowships. If one looks closely at the above judgments, it is obvious 
that each of these is very subjective; in fact it would be very difficult 
to find a set of objective criteria on which many colleagues could agree wtien 
judging journal publications, books, etc. Those who have read several advisory 
editors' reviews of their articles or experienqed the acceptance of a once 
rejected article by another equally reputable journal^ could attest to this 
subjectivity. 

Wliere then is the rationale for maintaining that the evaluation of 
teaching based on: (a) the testimony of department heads and deans, (b) 
comments of colleagues who are well acquainted with the teaching performance 
of the instructor, (c) achievement of students, (d) quality of teaching materials 
prepared by the instructor, and (e) the judgments of students exposed to the 
instructor, is more subjective, less reliable, and less accurate than the 
judgments of scholarly achievement? Apparently, the answer lies in the fact 
that the methods and techniques used in evaluating scholarly achievement have 
remained unchanged for so long that they have become accepted as standard 
without question or evaluation. In contrast, the evaluation of teaching has 
been subject to a wide variety of approaches with no commonly accepted methods 
or techniques except for student judgmi^nts gathered via rating forms, some of 
which have not, admittedly, been professionally designed. 
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A rpLOiiL r.Liuly l)y I aiUl rind l ipsol: (lO/!,)) furlhor oinpluis i ;?ns tJio nood for • 
ifisti.tntions lOtnLinv nM.:(»fin i z i luj nrid rowi\rtlii)q instructional orforLs. Tliny 
condnrt.ofi a survoy of U.S. facult-.y iitrMiihors and round that as a (jroup they are 
moro interostoil in toacliln*)' tlian rosoarch and r^chularsh ip. In fact, for ovory 
ono professor strongly dovotod to research, they found that there are nine 
cciiial ly .devoted to teachinci. 

ir institutions of higher education (and thefr departments) are serious 
about encouraging excellence in teaching, then they must be prepared to reward 
such excellence in a manner similar to that for excellence in scholarly achieve- 
ment. One approach tovard achieving this goal v/ould ue to have departments 
define v^hat v. :n5titbtes effe'jtive instruction and tnen establish a system of 
evaluation which I'.ou'Mj have as its two major outcomes: (a) cjuidance for improve- 
ment -and (b) rev^rd for s'tijcess. An integral element would be an agreement 
(or contract) betwe'^n Lh. rlf^pjruiient head and the faculty member outlining the 
percentage of finri' <: .\t intiiicnt to teaching, research, and service. The rest of 
this paper will be devoted to outlining one possible system of evaluation for 
encouraging instructional and faculty d velopii'ent (Aleamoni, 1976a). 
Methods of Assessing Instructional Effectiveness 

Evaluation of student attitudes about instructors and instruction is 
perhaps the most commonly used method of assessing instructional effective- 
ness. The rationale for this is that students are assumed to be the only 
ones who are constantly exposed to the elements of the, course (e.g., instructor, 
textbook, homework,' course content, method of instruction, etc.) and therefore, 
are the most logical evaluators of the quality and effectiveness of those 



(!l()iiUMit,s . In adfti t: iiifi, sIikIlmiL d U: i tuflf)?; c.in inil i(;(\t:(> .iruas of rapport, ilornMior. 
of tiomiiiunicntion ov (vxistcMico oT ijrobloiDs <md t.horobv h(Mp irv;. trncCnrs tis 
W(v.|l lU; iiistnicLional ilovi/lopniunt personnel doscribo iind defino t;ho loarivin'i 
(Miv ironiiion t iiioro concroLoly and oljjort:iv(»]y than tln»y coDld t.brnuqfi othor 
types ot nioasiironient'.s (Aloanioni ft Sponcor, 1973; Aloaiiioni, 1978). 

Sol f-evaluatioii on the part: of tho faculty iiiomber is nocossary in order to 
dot(?ni)ino the rationale for the various olomqnts of tho course(s} t.aught. Such 
scl I -ovtHuation could deal with: (a) course content, (b) method of instruction, 
(c) ability to keep student interest and attention high, (d) ability to proinot4 
student learning, (e) course and instructional objecti\/es, and (f) course and 
instructional organization. 

The quality of student learning is another important variable to consider 
and could be used to>determine if the course objectives are being met. It 
would be possible to get a measure of student achievement 'in a course by 
employing course or departnental ly agreed upon examinations that were designed 
to fit the instructional objectives. Nationally standardized examinations 
could also be used if they were judged to accurately measure the satisfaction 
of course and/or departmental objectives. Each of these could then be used either 
as final examinations or as a part of the final examination. Other measures 
could be obtained by fol lowup' measurements of student performance in succeeding 
courses or by following the students after they have left the institution and 
recording their performance at other institutions of higher education or in 
their jobs or professions. 

Faculty (colleague), and department head attitudes and judgments represent 
the most comprehensive method of assessment. For example, these judgments could 
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hi» Ijir.fft upon 'an cvd I ua I. ion ()\ : (d) Uic iirocodmMl nih'qii«\ry of t:ho t:oursp('0» 
(I)) ..)|.)|»ro|»r ial,iMu»s'; nt' tin* com^^ip coiil.rnU (i;) Mm ()ppr(i|U' i.i l:rn(*',s of" Ihn 
i:om'''i(? ohjortivi^M (^^ tln) iUti!(|U(i(\y aod (|iMlit:y of t.ho rnin^so oxatniiwit. ions, (o) 
tlio sMhility of an i ns ttMu: for ' s toachhuj offor, t.1 vonoss ov(n^ t;h(^ yoa)'s, (I) tin' 
stU(le?it otti tud ini\l ropoi^t, ((.)) t:t)o instructor's sol f-ropoi^t, (h) tlio nua lity 
or stu(li?nt loarnincj, and (i) othor course niatqrials oi' coininonts which tho 
instructor wishes to submit. Such ovaluations ideally should bo niado by throo- ' 

In 

porson dopartinen tal roviow committeos usincj v^/ol 1 -structured cjuidolines and criteria 
of excellence as well as using instructional development personnel expertise 
where needed. • : 

FeGdbacJ<^ to^ tjie Instru ctor 

All assessment results should^ be provided to the faculty member in a form 
which would allow easy and accurate identification of instructional problems 
as well as suggestions for solving s^uch problems. After such results have 
been carefully analyzed and possibly used by the faculty member to. design and 
implement solutions to instructional problems, then they could be submitted to 
the department head and weighted appropriately in a "teaching research + 
service" equation for use in rank, pay, and tenure considerations. 

The student attitude or rating data should be presented to the instructor 
In an easily readable and interpretable format. Student responses to the 
questionnaire items should be reported in summary form for each course and 
appropriate comparisons should be made between the mean scores in^the instructor's 
course (by each item and groups of items) and means for^similar instructors 
teaching similar courses in the same department, college, and university. For 
examples of appropriate normative ^mparisons see Costin,^ Greenough, and Menges 
(.1971) and Aleamoni (1972a, 1976b, 1979). " . 
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111,, ins |.riii;l.i))> !;||oultl Im' iiroviJu'd wMli ^idil i I. i on.i I iiil.arptMil.ivc iii.il i.i I 

.r. wi'll a;, Willi llic |H)'.'. ilii 1 i l.y ol cohmiI l.a I: hui with llu-' i iisl.nic t: imid 1 ilovo 1 npni 

■I 

I I'c'.pon', ililc iVir pr(iv|(llii>i /.Liidcnl; <\1.M l.inl in.il iiironii.i Unii. Aiiv 
,u|(li l.idiia 1 I'Viiliial.ivo infuniiatloii ami t'osmn'cn', noodofl' i ii llii'; area <Am\\^\ al-.o 
I),. ,li«.(;ir.'a;d with t.hu i ii'Ui'iu; t, I ona I dovt^lopiiUMi I. pi^rsnnmil . All inl.et'prol.lvi- 
111,1 l.<0'i a rand connnUal. ion ••.honid coiindii I.im to on Idonti ry lrm iiist.nicf.loMa l 
-it.rumjl.li". and woakm.'ssos a^nd LhtMi aiji'ooiiKi on what, should bo doiio to roiiiody 
(.lie wcakiio:;'~,(-".> . 

Iho quality of r,tudoiit lenniiiu] data (|ather«-/d :,hould also bo prosentod 
to the instructor in an oabiily readablo and i n terpi'otabl n ropo't. This report, 
however, should concentrate on the achievement of c>.ursu, departnient, college 
cind univorsity instructional objectives. 

Fdculty (colleague) and dopariiiient head attitudes and judqnitnts should 
be presented to the faculty member in an easily readable and interpretablo 
report. The report should outline the committee and individual judqments 
iiiado on each element of the instructional setting considered and identify 
the strengths and weaknesses observed. Construct4ve criticisms that would 
'allow the instructor to plan a strategy for improvement of the areas of weak- 
viess should appear in the report. Personal consultation should take place 
between the department head and faculty member concerning all aspects of this 
report. The department head should explain the options or resources available 
"to instructors to help them in their attempts to' improve their instructional 
weaknesses . - ^ ' 

' 'The department head should hold individual conferences with instructors 
concerning all the evaluations that have been conducted and what they mean 
to the instructor. (This, of cpffrse, assumes that the department head under- 
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sLainl'. Uio iWii liiii I. i<uis iM.nl<).) MiM i(np,)r IiimmiI. ImsiiI shiiiiltl (Miiph-is i /i^ wImI. IIm^ 
instructor iiiUMt'. l.o do in oyi\o\ l.n rt^'Mry t.hc wisikmviSrN nr iirohlc'iiis uncnynro*! 
ill {|i<' vririDiri rvaliiuMons mAt^ lln* <lrp.irtiii(Mil. hisul '.hoiiM -ilso cltU'ity how 
MHii iwa li]<i t. ion . .iml iMtort/i r(in lu» u'o'ti In lUr y\ni(,, |hi,v, ,ui(l tiMHUM^ coivJtloraU 
ill U)o Intl i V iitua I t'io:n I i,y nionihiM's . * 
'OpUon'. for llio 1 in, t.nic t'or 

Whnn, (1 1 1 t h(» ro:.';n'iMiUM)t,s oT inst.rinM.iona t (M t oc U vcmkv^.s (\ro highly |uv;iUvo 
tlion tli(» instructor coiihl recpu^'.t Llu^y ho (ilticiMi in his/hor (l(»()artiiionta 1 fih* 
to ho us(mI for laiik, \)i\yi and toniitH) cons idorat ion, If, on tho othor liand, all 
t)\<) assossiiH-nts of Instructional ot'focl.iviMu^ss aro not highly positive, thon 
tho instructor should idontify (with tho possible aid of instructional dovolopmon 
porsonnol as woll as the department head) what needs to be done to improve the 
instructional weaknesses. The instructor could then take the necessary steps 
Xo rectify those instructional weaknesses. However, if additional resources . 
are needed to' plan and implement an instructional improvement strategy, then 
a proposal could be generated along the following lines: 

1. The instructor would draft a proposal outlining what needs to be 
done, what instructional resources will be needed, how long it will take, how 
"much it will cost, and how it will be evaluated. ' ' ' 

2. The instructor would discuss the proposal with the def)artment head and 
obtain any necessary approval and/or additional resources. 

3. The instructor would also discuss the proposal with the appropriate; 
instructional development personnel to determine what services are available. ' 
in trying to implement the proposal . " , " 

4. Finall-y, the instructor would modify the proposal according to the 
suggestions made in discussions with the 'department head and .instructional 
development personnel, 

<f 11 ^ 



liii|i|,.iih>iilal.ii)n ni llu' liislriirl iiiii.il Iniiiri'Vpnu'ii I l'rn|„i',.il ,>ii.M lin iTdi'": i ' 
I JimI l i'|itirl. . 

OiiM' ,1 liiMl Ill I. Ill' |ii'n|in',,i 1 is •icn.'ral^"i .huI ,i | L .ipprnv.i 1 <iIiIsui)»m| 

r.^'.Hijrtcs itlriii ifir«l, the iii',triii:lnr shoiiM iiiiu.iit! llu.' i im.M.il.l.' iit'Md.'.l 
1(1 iiiiph'HiH'nl, I lit' |iisipi)'Mi I . 

At l.iT I. Ill' 111', Ij'iir. I, iniiiil liii|)i'6vi'iiii'iil |ii'ii,li'i:l h.i', lii'i'ii .:iiiii|i |i>l iMl -uul all llii> 
,.'/,(|iMl.lri«i \h\'. liOL'ii iloiio, Iho iiisl.riiclny shoiiUI iin'p.W'i) .i i iinippclK.'ii'i I vn I IimI 
i,,,„,ri. lii',lri|i;l-luii.il tlcvc lupiiii'ilt |M'r'.oiiiii' 1 .:inilil he usimI .iI. Iliis imllil. il 
lu'cd.'ij. Ilu) rinal^ ropni't. (contain inq i-ivuilLs of Lho in-, inic'tloiia I Improvo - 
iiicnl. criot'l.) anil tlio propotiaV sliniild then lio '.iilniil Llnl t,n Ihc dopartDiont. ii>a(l 
.111(1 placdd 1n LIu! i ii'. Lmic Lor ' !i tiopar tmcnto 1 Cilc fur u'\o In raiik^, pay, and 
tenure' con-vide'ra 11^11'^. 

Adiiiinistra t.iye I'rocodiiro' fin' ivC'i'ijiierj mj lnsl-.rui;Lion ij] the Ins M LiilJ (ina I, Rnward 

Svsteni 

...w. — — 

All of th^'^ instructiunal ef fee ti venos<^ as!^.or.sii)onts should ho evaluated 
and consiaor(^d rolalive to the teaching loud ro(|uirod of the faculty member. 
If an instructional improvement proposal and final )'oport art in the instruc- 
tor's departmental ' file, then they should be'ana'lyzed carefully and evaluated 
By a departmental review^ corrmi ttee Utilizing methods and techniques developed 
by the dep'artment. Thi.s evidence, should be considered p)^di)ortionate to: ^ ^ 
(d) the teaching load required of the faculty member, and *(b) the time needed 
to generate the proposal, complete^e project, and write up the final report. 
That evidence should be weighted acTcordingly in the "teaching + research + 
Service" equation in arriving at rank, pay, and tenure recommendations. . 

In conclusion, the higher level administration of the institution (including 
deans and department heads) should encourage the development and use of such a 
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system ivs an integral part of the institutional reward scheme. When such 
"systems" have been developed, the satisfaction on the part of the participant 
faculty and administrators appears to be very high (Harrell, 1979; O'Connell & 
Sinartt, 1979). It, therefore, seems logical to encourage the development of 
comprehensive systems of instructional-evaluation that have guidance for improve- 
ment and reward for success as their two major outcomes. Not to do so may 
suggest a lack of interest in and a lack of concern with instruction and 
insti^uctional improvement. 
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